The Practical Value of Field Experiments

In recent years, many firms have had greater opportunities to conduct field experiments to help improve their marketing decisions, but the practical value of using field experiments to optimize marketing decisions remains relatively unstudied. We investigate advertising and promotion decisions that require estimating a large matrix of cross-product demand elasticities and ask: how many experiments are required as the number of products grows? Our main result demonstrates that if we add structure we can learn faster and reduce the number of experiments that are required. The number of experiments required may grow just logarithmically with the number of products. These findings potentially have important implications for the application of field experiments. Firms may be able to obtain meaningful estimates using a practically feasible number of experiments, even in settings with a large number of products. The relevance of our findings is confirmed through simulations using realistic parameters gathered from a real-world large-scale pricing experiment. We also discuss an application of our model to making product line pricing decisions, together with other types of marketing decisions.